Ontological test of the IAT 1 Running Heading: ONTOLOGICAL TEST OF THE IAT An ontological test of the IAT: Self-activation can increase predictive validity

Extensive research has been conducted demonstrating the predictive validity and reliability of the IAT for a broad array of behaviors and contexts. However, less work has been done examining its underlying construct validity. This contribution focuses on examining whether a core theoretical foundation of the IAT paradigm is valid, specifically, whether the IAT effect draws on the Social Knowledge Structure. We present four studies within different domains that show that the IAT does indeed appear to draw on the SKS. The data show that activation of the self before the categorization task enhances the predictive validity of the IAT, as one would expect if the IAT reflects the SKS. We discuss theoretical reasons for these findings, with emphasis also on underlying statistical/psychometric issues. Ontological test of the IAT 3 Implicit measures have achieved a prominent status in psychological research in the last few years. The Implicit Association Test (IAT) represents the most popular of these measures. Since the original paper in which the IAT was introduced (Greenwald, McGhee, & Schwartz, 1998), dozens of studies have applied the paradigm to an impressively diverse array of issues (for a review, see Poehlman, Uhlmann, Greenwald, & Banaji, 2006). There have been relatively fewer attempts investigating the mechanisms underlying the IAT. Moreover, most of the research addressing this issue has focused on the specifics of the IAT effect (Rothermund & Wentura, 2001), on the influence of confounding effects in the IAT score such as words familiarity (Dasgupta, McGhee, Greenwald, & Banaji, 2000) and extrapersonal factors (Olson & Fazio, 2004a), and on the impact of contextual effects on IAT scores (Mitchell, Nosek, & Banaji, 2003). In this contribution we will examine one of the basic assumptions of the IAT, namely that it reflects associative links in the Social Knowledge Structure (SKS; Greenwald, Banaji, Rudman, Farnham, Nosek, & Mellot, 2002). The SKS Assumption in the IAT The IAT is a double discrimination task used to measure the relative strength of the associations between pairs of concepts. Even though it is a relatively new paradigm, it has rapidly become a widespread tool in social psychological research. The IAT relies on the assumption that, if a target concept and an attribute concept are highly associated (congruent), the task will be easier, and therefore quicker, when they share the same response key than when they require a different response key (for procedural details, see Greenwald, McGhee, & Schwartz, 1998). The theoretical basis of the IAT, and one of its most important assumptions, relies on its tapping into the SKS. The SKS is a network of variable-strength associations that correspond to social Ontological test of the IAT 4 psychological concepts (self-concept, self-esteem, stereotype, attitude) and attributes (Greenwald et al., 2002, p. 5, Figure 1), presumably stored in long-term memory. The self is a central entity in the SKS. This centrality is represented by “...its being associated with many other concepts that are themselves highly connected in the structure” (ib., p. 5). The assumption of the SKS is central to the logic of the IAT. It provides both a theoretical foundation and a rationale for its capability to predict behaviors. From a theoretical point of view, the SKS represents the link between measure and concepts. An attitude towards an object is a stored evaluation in memory, relatively stable over time, and can be activated automatically (e.g., Fazio, 1990). The SKS therefore represents a theoretical bed that accommodates the view of attitudes as associations between objects (actions, groups) and valence. From a predictive point of view, an IAT should predict some germane behaviors if it genuinely reflects personal associations between the relevant target and valence. There is at least one sense in which this is an essential requirement. Suppose that the IAT simply reflects specific mechanisms underlying the cognitive operations activated by the task. The ranking of individuals in the resulting IAT score should therefore be affected only by individual differences in the operation of such mechanisms (method variance), for instance, stimulus-response compatibility or task-switching costs (e.g. Mierke & Klauer, 2003). However, it is unclear how individual differences in task switching costs, for example, could predict specific behaviors such as condom use or food choice. To sum up, a focus on predictive validity appears not only informative about the pragmatic value of the IAT, but also important in terms of its theoretical foundations, namely the SKS assumption. The Mechanism of Self-activation Ontological test of the IAT 5 Self-activation can be defined as the cognitive activation of any kind of selfrelated knowledge; it does not necessarily require conscious awareness, and it is characterized by a general heightened state of accessibility of self-related knowledge (Stapel & Tesser, 2001, p. 743). The constructs of self-focus and self-awareness are often used interchangeably with the concept of self-activation. It has been proposed that they should be distinguished from self-activation, mainly because the latter does not require reflective conscious self-attentiveness (Stapel & Tesser, 2001, footnote 1, p. 743). While the argument put forward appears compelling, an inspection of the actual use of the terms in published research produces a far less clear picture. Experimental manipulations that would seem indistinguishable in terms of conscious activation, are labeled as self-focus, self-awareness, or self-activation manipulations in different contributions (e.g., Dijksterhuis & Van Knippenberg, 2000; Macrae, Bodenhausen, & Milne, 1998). Here we will refer to the term self-activation in its generic sense and sidestep these more subtle differences that in practice seem as much real as a question of semantics. Self-activation typically results from an experimental manipulation that renders self-related cognitions especially salient or accessible. The consequences of this increased accessibility can be diverse. For instance, it has been shown that selfactivation increases social comparison (Stapel & Tesser, 2001), the efficiency of selfregulatory processes (Carver & Scheier, 1981), the attitude-behavior consistency (Pryor, Gibbons, Wicklund, Fazio, & Hood, 1977), and decreases stereotyping (Macrae, Bodenhausen, & Milne, 1998). On a different but relevant stream of research, it has been shown that individuals high in self-consciousness, characterized by chronically higher accessibility of self-related knowledge, are particularly sensitive to experimental manipulations such as subliminal priming used to instigate automatic Ontological test of the IAT 6 non-conscious behavior (Hull, Slone, Meteyer and Matthews, 2002). Whilst the specific consequences of self-activation can be diverse and also probably influenced by the demands of the subsequent task, the mechanism is pretty much the same. Selfactivation increases the accessibility of self-relevant thoughts and constructs. This affects subsequent activities in the direction of the thoughts and constructs that are momentarily more accessible. Testing the SKS Assumption via Self-activation The links between the IAT, the SKS, and self-activation should be apparent at this point. To summarize, the IAT is a task of which the outcome depends on the difference in speed of the motor actions (i.e., movement of the index finger) needed to categorize correctly and which is reflected in different response latencies. This response speed is critically influenced by the relative ease or difficulty in activating the necessary motor command. This relative ease or difficulty, in turn, depends on the strength of the associations between the two pairs of concepts that might interfere or facilitate the use of the same response key. The strength of these associations is reflected in the SKS. Self-activation increases the accessibility of self-related thoughts and concepts. Therefore, it should follow that an IAT completed immediately after a self-activation manipulation should better reflect the SKS and, as a consequence, an IAT score so obtained should be more predictive of actual behavior, as it contains a relatively greater proportion of valid variance (variance that reflects the SKS). Overview of the Studies In sum, our key hypothesis is that self-activation should increase the predictive validity of the IAT. This prediction relies on an important assumption underlying the IAT, namely that it reflects concepts and valences as stored in the SKS. We present four studies that test this hypothesis. Study 1 concerns attitudes toward alcohol, Study Ontological test of the IAT 7 2 focuses on attitudes toward academic disciplines and includes students from Arts and from Science departments, Study 3 examines attitudes toward junk food, and Study 4 is about attitudes towards Americans. The key criteria to be predicted are self-reported behaviors (Study 1 and 3), group membership (Study 2) and actual behaviors in the form of judgments (Study 4). The results show that across attitudes, behaviors, and manipulations, self-activation increases the predictive validity of the IAT. Study 1: Alcohol Drinking alcohol is a relatively common behavior. The most recent national survey in the United Kingdom estimated that adults aged 14 and over drink on average 11.3 units of alcohol per week (Institute of Alcohol Studies, 2005). Some studies have applied the IAT (or modifications of it) to the issue of drinking alcohol with promising results (for a review, see Wiers, Houben, Smulders, Conrod, & Jones, 2005). For instance, Wiers, Van Worden, Smulders, and De Jong (2002) found a significant relationship (r=.37) between a standard valence IAT and a composite index of alcohol use. Typically, the studies have focused on predicting some kind of composite index of alcohol consumption rather than a consumption index of alcohol relative to soft drinks, even when the IAT measure has been defined using a contrast category of soft-drinks. Therefore, whereas there is some empirical evidence of predictive validity of an IAT for alcohol consumption, little is known to what extent it can predict a relative preference over soft-drinks consumption. The first study explores this issue and tests the key hypothesis that self-activation will increase the predictive validity of the IAT. Method Participants. The sample consisted of 60 participants, 27 males and 33 females, Ontological test of the IAT 8 with an average age of 26.2 years (SD=5). Of these, 48 (80%) were successfully recontacted by e-mail after one week to obtain a second behavioral measure. The participants were predominantly university students and were contacted on campus or through informal networks. Design and Procedure. The design was a simple 2-condition between-subjects factor. Participants were told that they would be completing two experiments and were randomly allocated to either a self-activation or a neutral condition. Each participant was tested individually with a laptop in different locations. Care was taken that during the experiment no external distractions or noises were present. The first experiment was presented as a pilot study on proofreading and word-search whereas the second was a study about their preferences towards different types of drinks. The first experiment was actually the self-activation manipulation, modeled after Brewer and Gardner (1996) and used in other studies on self activation (Stapel & Tesser, 2001). Participants were asked to read paragraphs describing a trip to a city and to circle certain words within two minutes. The text was identical, but the words to be encircled were different in the two conditions. In the self-activation condition, the words were “I”, “me”, “my”, and “myself”, whereas in the neutral condition the words were “the” and “a”. In both cases there were 19 such words. Participants were then asked to perform the next tasks at a laptop with a 14.1-inch display set at a resolution of 1024 x 758, color depth set at 16 bit and refresh rate at 72Hz. The tasks were programmed with Inquisit (version 1.33). First, participants completed an IAT on alcohol vs. soft drinks. Our implementation of the IAT followed the established format of seven steps (cf. Greenwald, McGhee, & Schwartz, 1998; Greenwald, Nosek, and Banaji, 2003). The target category pairing was Alcoholic drinks (beer, wine, whisky, lager, cider) and Ontological test of the IAT 9 Soft drinks (coke, pepsi, fanta, sprite, juices) whereas the attribute categories were Pleasant (happy, smile, joy, peace, pleasure) and Unpleasant (pain, death, poison, agony, vomit). There were 20 practice trials for the non-critical steps (steps 1, 2, and 5), 20 steps for the training stage of the critical pairs (steps 3 and 6), and 60 trials (plus two dummy initial trials that were discarded) for the critical steps 4 and 7. The stimuli were presented in a random order for all participants. The order of steps 3-4 and 6-7 was fixed for all participants, with Alcoholic drinks paired with Pleasant in step 3-4 and with Unpleasant in step 6-7. Participants were asked to press the left key (letter d) or the right key (letter k) depending on the category of the stimulus. An error message consisting of an acoustic beep was delivered upon incorrect classification. The inter-trial interval was 400 ms. Participants were then asked their explicit attitude, first, towards drinking alcohol and, then, soft drinks. They responded to the stem “I think that to drink alcohol (soft drinks) is for me:” followed by 7 semantic differential pairs of adjectives (bad-good, foolish-wise, unpleasant-pleasant, negative-positive, unenjoyableenjoyable, unhealthy-healthy, unattractive-attractive) on a 7-point scale. Next, participants completed a self-reported behavioral grid asking them to report how many units of alcoholic and soft drinks they usually consumed for each day of an average week. The concept of a unit of alcohol is commonly used in the UK and corresponds to specified approximated quantities of different types of alcoholic drinks. For instance, one unit of alcohol corresponds to a small, 125 ml. glass of wine, half a pint (i.e., 284 ml) of beer/cider/lager, and a standard measure (25 ml.) of spirits (e.g., whisky). To further reduce idiosyncratic reporting, a small legend reported the units corresponding to each alcoholic and soft drink. The list of alcoholic drinks included beer, wine, lager, spirits, cider, alcopops, and other alcoholic drinks, whereas Ontological test of the IAT 10 the list of soft drinks included Coke/Pepsi, lemonade, juices, and other soft drinks. Finally, participants were thanked for their participation and were informed that there would be a brief final part of the experiment in one week. They were asked for an email contact address. After one week, participants were sent the previously described self-reported drinking grid and asked for their drinking behavior in the previous, rather than an average, week. Data analysis strategy. The same data analysis strategy was used in the four studies. We first inspected the psychometric properties of the measures and report relevant descriptive aspects of the data. Next, a regression approach was adopted, centering variables before calculating interaction terms to reduce unessential multicollinearity (Cohen, Cohen, West, & Aiken, 2003). To test the key hypothesis of increased predictive validity of the IAT under self-activation manipulation, we ran regressions in which each dependent variable in the study was predicted by the IAT score, the experimental condition (dummy coded as 0=control and 1=self-activation), and their interaction. The first order effect term for the IAT would reflect the slope of the regression line in the control condition. A positive significant interaction term would signal a successful test of the hypothesis, indicating that the IAT score has higher predictivity in the self-activation condition. The interaction was further probed by reversing the dummy coding to inspect the effect of the IAT in the self-activation group (Aiken & West, 1991). To establish whether the hypothesized effect was unique to the IAT, three additional sets of regressions were conducted. The first set tested whether the same effect was present for explicit attitudes. The independent variables were therefore the explicit attitude score, the experimental condition, and their interaction. The lack of a significant interaction term in the latter regression set would signal that self-activation Ontological test of the IAT 11 works uniquely with the IAT. The second regression set tested the possibility that the interaction between the IAT and the self-activation condition is due to the shared variance between the implicit and the explicit measures. If that is the case, the effect should vanish if the explicit attitude measure is included in the equation. In other words, we tested whether the moderation effect is mediated or suppressed by the explicit attitudinal measure. Finally, the third set tested the specific issue of whether self-activation increases the correspondence between implicit and explicit measures of attitude. The regression therefore included the explicit attitude score as the dependent variable and the IAT score, the experimental condition, and their interaction as the independent variables. The lack of a significant interaction term would suggest that self-activation does not simultaneously increase the salience of propositional (i.e., explicit) and associative (i.e., implicit) associations. Taken together, once the presence of a significant effect of self-activation on the predictive validity of the IAT is established, these three additional analyses should clarify the extent to which the self-activation manipulation works primarily or uniquely at an implicit level. Results and Discussion The IAT score was calculated with the algorithm D (deletion of latencies below 400ms, errors replaced with the mean of the correct responses plus 600ms) developed by Greenwald, Nosek, and Banaji (2003), included all the 80 trials (20 practice and 60 test), and was calculated such that the practice and test stages had a weight proportional to the number of trials included in each (in this case 25% and 75%, respectively). The reliability of the IAT score was good (α=. 80). It was obtained by calculating 80 IAT scores (one for each pair of trials) and using them as items. The IAT score was computed such that higher scores expressed an implicit preference towards alcoholic over soft drinks. The attitude score was calculated as the difference Ontological test of the IAT 12 between the sums of the semantic differentials, with positive scores indicating a preference for alcohol over soft drinks, and showed good reliability (α=. 90). The correlation between implicit and explicit attitudes was not significant (r=.16, p=.234). The means of the measures for the two groups (self-activation vs. control) for all four studies are reported in Table 1. [Insert Table 1 about here] The groups did not differ in their explicit (t(58)=0.45, p=.652) and implicit attitudes (t(58)=0.04, p=.970). Mean units of drinks in an average week and in the last week varied between 8.9 (soft drinks, average week) and 11.7 (alcohol, average week). The two groups (self-activation vs. control) did not differ in terms of drinking behavior (all p’s >.45). These results suggest that the assignment was effectively random. Two indices of relative preference for drinking alcohol (positive values) or soft drinks (negative values) was calculated by subtracting the total amount of units of soft drinks from those of alcohol, both for an average week and the previous week. The two indices were correlated significantly (r=.52) and aggregated in an overall index of relative alcohol consumption. The multiple regression to test the key hypothesis explained 21.3% of the variance, with a significant effect of the experimental condition (β=.58, p=.039) crucially qualified by the expected significant interaction (β=.57, p=.029). The IAT was not a significant predictor in the control condition (β=-.09, p=.586) whereas it significantly predicted the drinking alcohol index in the self-activation condition (β=.48, p=.017). The two simple slopes are presented in Figure 1. [Insert Figure 1 about here] A second set of regressions ascertained that the effect was not present for explicit attitudes. The drinking index was significantly and strongly predicted by Ontological test of the IAT 13 explicit attitudes (β=.54, p<.001), but the interaction term with the experimental condition was not significant (β=.18, p<.459). The third set of regressions showed that explicit attitudes mediated or suppressed the self-activation effect on the IAT. In fact, the inclusion of the explicit attitudes as a predictor (β=.47, p=.001) rendered the interaction term IAT x Experimental condition no longer significant (β=.30, p=.209). Finally, we tested whether the correlation between IAT and explicit attitudes changed as a function of the self-activation condition. The results showed that the interaction term involving IAT and experimental condition was significant (β=.58, p=.033). IAT and explicit attitudes were not significantly correlated in the control condition (β=-.09, p=.596) and significantly associated in the self-activation condition